Transactional access to records on secondary storage in an in-memory database

ABSTRACT

Various technologies described herein pertain to accessing records of an in-memory database. A memory can include a primary storage and a data store can include a secondary storage. The primary storage includes a first subset of the records of the database, and the secondary storage includes a second subset of the records of the database. Moreover, the memory includes an update memo that includes timestamp notices. The timestamp notices specify statuses of records of the database having updates that span the primary storage and the secondary storage. The memory further includes a database management system that is executable by a processor. The database management system processes a transaction. Further, the database management system includes a storage interface component that accesses a record of the database for the transaction based on at least one of the timestamp notices of the update memo.

BACKGROUND

Traditional database management systems were designed with data beingdisk resident (e.g., store data on secondary storage); accordingly, inthese systems, data is paged in and out of memory as needed. Morerecently there has been a shift in design of database managementsystems, such as online transaction processing (OLTP) databases,attributable at least in part to decreases in memory costs. Accordingly,several database management systems (e.g., main memory database systems)have emerged that primarily rely on memory for data storage (e.g., mostor all of the data may be stored in memory as opposed to secondarystorage).

In a transactional workload, record access patterns tend to be skewed.For instance, some records are “hot” and accessed frequently (e.g.,these records can be considered to be included in a working set), whileothers records are “cold” and accessed infrequently, if ever.Performance of database engines can depend on the hot records residingin memory. Further, cold records can be moved to secondary storage withless impact on overall system performance.

SUMMARY

Described herein are various technologies that pertain to accessingrecords of an in-memory database. A memory can include a primary storageand a data store can include a secondary storage. The primary storageincludes a first subset of the records of the database, and thesecondary storage includes a second subset of the records of thedatabase. Moreover, the memory includes an update memo that includestimestamp notices. The timestamp notices specify statuses of records ofthe database having updates that span the primary storage and thesecondary storage. The memory further includes a database managementsystem that is executable by a processor. The database management systemprocesses a transaction. Further, the database management systemincludes a storage interface component that accesses a record of thedatabase for the transaction based on at least one of the timestampnotices of the update memo.

The above summary presents a simplified summary in order to provide abasic understanding of some aspects of the systems and/or methodsdiscussed herein. This summary is not an extensive overview of thesystems and/or methods discussed herein. It is not intended to identifykey/critical elements or to delineate the scope of such systems and/ormethods. Its sole purpose is to present some concepts in a simplifiedform as a prelude to the more detailed description that is presentedlater.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a functional block diagram of an exemplary computingsystem that includes a database management system.

FIG. 2 illustrates a functional block diagram of an exemplary systemthat accesses and migrates records retained in a primary storage and asecondary storage.

FIGS. 3-4 illustrate an example of storing, accessing, and updatingrecords without separating the records between the primary storage andthe secondary storage.

FIG. 5 illustrates an exemplary structure of a record in the secondarystorage.

FIG. 6 illustrates an exemplary structure of a cached record in aprivate cache.

FIG. 7 illustrates an exemplary structure of a timestamp notice in anupdate memo.

FIG. 8 illustrates a functional block diagram of an exemplary systemthat performs various operations in connection with use of the secondarystorage.

FIG. 9 illustrates an exemplary effect of migrating a record from theprimary storage to the secondary storage.

FIG. 10 illustrates an exemplary effect of deletion of a record from thesecondary storage.

FIG. 11 is a flow diagram that illustrates an exemplary methodology ofmigrating a record from a primary storage to a secondary storage.

FIG. 12 is a flow diagram that illustrates an exemplary methodology ofreading a record of a database as part of processing a transaction.

FIG. 13 illustrates an exemplary computing device.

FIG. 14 illustrates an exemplary computing system.

DETAILED DESCRIPTION

Various technologies pertaining to transactional access to records onsecondary storage in an in-memory database are now described withreference to the drawings, wherein like reference numerals are used torefer to like elements throughout. In the following description, forpurposes of explanation, numerous specific details are set forth inorder to provide a thorough understanding of one or more aspects. It maybe evident, however, that such aspect(s) may be practiced without thesespecific details. In other instances, well-known structures and devicesare shown in block diagram form in order to facilitate describing one ormore aspects. Further, it is to be understood that functionality that isdescribed as being carried out by certain system components may beperformed by multiple components. Similarly, for instance, a componentmay be configured to perform functionality that is described as beingcarried out by multiple components.

Moreover, the term “or” is intended to mean an inclusive “or” ratherthan an exclusive “or.” That is, unless specified otherwise, or clearfrom the context, the phrase “X employs A or B” is intended to mean anyof the natural inclusive permutations. That is, the phrase “X employs Aor B” is satisfied by any of the following instances: X employs A; Xemploys B; or X employs both A and B. In addition, the articles “a” and“an” as used in this application and the appended claims shouldgenerally be construed to mean “one or more” unless specified otherwiseor clear from the context to be directed to a singular form.

Referring now to the drawings, FIG. 1 illustrates a computing system 100that includes a database management system 108. The computing system 100includes a processor 102 and a memory 104 (e.g., main memory). Moreover,the computing system 100 includes a data store 106 external to thememory 104. The data store 106, for example, can include disk drive(s),flash-based solid-state drive(s) (SSD), other types of storage devices,a combination thereof, and the like. The processor 102 is configured toexecute instructions loaded into the memory 104 (e.g., one or moresystems loaded into the memory 104 are executable by the processor 102,one or more components loaded into the memory 104 are executable by theprocessor 102, etc.). As described in greater detail herein, the memory104 includes the database management system 108 that interacts with adatabase to capture and analyze data. Thus, the database managementsystem 108 is executable by the processor 102. Moreover, the databasecan be an in-memory database.

According to various examples, the computing system 100 can be orinclude a computing device. Pursuant to various illustrations, thecomputing device can be a desktop computing device, a mobile computingdevice (e.g., a laptop computing device, a mobile telephone, a tabletcomputing device, a wearable computing device, a handheld computingdevice, a portable gaming device, a personal digital assistance, etc.),a gaming console, a set-top box, an in-vehicle communications andinfotainment system, or the like. In accordance with other examples, thecomputing system 100 can be or include one or more server computingdevices. For instance, the computing system 100 can be or include one ormore datacenters, where a datacenter includes a plurality of servercomputing devices. Additionally or alternatively, the computing system100 can be a distributed computing system.

The memory 104 further includes a primary storage 110, and the datastore 106 (external to the memory 104) includes a secondary storage 112.As noted above, the database management system 108 interacts with adatabase, which includes records. The primary storage 110 and thesecondary storage 112 are separate portions of the database. The primarystorage 110 includes some records of the database (referred to herein asfirst records 114, a first subset of the records of the database), andthe secondary storage 112 includes other records of the database(referred to herein as second records 116, a second subset of therecords of the database). Collectively, the first records 114 and thesecond records 116 are referred to herein as records 114-116. Therecords 114-116, for example, can be rows of tables declared asin-memory tables (e.g., memory-optimized tables, etc.).

The database management system 108 (e.g., a database engine) creates,reads, updates, and deletes the records 114-116 of the database. Thedatabase management system 108 can be used with online transactionprocessing (OLTP) workloads. OLTP workloads oftentimes exhibit skewedaccess patterns where some records are frequently accessed, while otherrecords are infrequently or never accessed. As referred to herein, a“hot” record is a record that is frequently accessed and a “cold” recordis a record that is infrequently (or never) accessed.

The database includes at least one in-memory table managed by thedatabase management system 108. The database management system 108 cansupport some tables of the database being declared as in-memory tablesand other tables of the database not being in-memory tables. Recordsdescribed herein (e.g., the records 114-116, versions of records, etc.)refer to records of tables declared as in-memory tables managed by thedatabase management system 108. Accordingly, while not described herein,it is contemplated that the primary storage 110 and/or the secondarystorage 112 can additionally include disparate records of the othertables of the database that are not in-memory tables.

For some conventional database management systems, an in-memory table isto reside entirely in memory. However, even a frequently accessed tablemay exhibit access skew, where a fraction of records of the table arehot while other records of the table are cold. Storing the cold recordsin memory can be more costly as compared to storing such records in adata store. Moreover, performance may be degraded if the cold recordsare stored in memory as compared to being stored in a data store.

In contrast, the database management system 108 includes a storageinterface component 120 that supports automatically migrating the coldrecords to the secondary storage 112, while the hot records remain inthe primary storage 110. The separation of the records 114-116 into theprimary storage 110 and the secondary storage 112 is visible to thestorage interface component 120, while upper layers of the databasemanagement system 108 (and applications that interact with the databasemanagement system 108) are unaware of such separation; thus, whether theprimary storage 110 includes a given record or the secondary storage 112includes the given record is transparent to the upper layers of thedatabase management system 108 (and the applications). The upper layersof the database management system 108 (and the applications) can beunaffected by the split of the records 114-116 between the primarystorage 110 and the secondary storage 112.

The storage interface component 120 automatically and transparentlymanages the first records 114 retained in the primary storage 110 andthe second records 116 retained in the secondary storage 112. Thestorage interface component 120 can access the first records 114 and thesecond records 116. For instance, the storage interface component 120can create, read, update, and delete the first records 114 and thesecond records 116. The storage interface component 120 can furthermigrate records between the primary storage 110 and the secondarystorage 112 (e.g. the storage interface component 120 can migrate arecord from the primary storage 110 to the secondary storage 112, thestorage interface component 120 can migrate a record from the secondarystorage 112 to the primary storage 110).

The primary storage 110 supports transactions. The secondary storage 112may, but need not, support transactions. While the secondary storage 112need not support transactions, the storage interface component 120 canaccess, update, and migrate the second records 116 in the secondarystorage 112 (e.g., records can be migrated to and/or from the secondarystorage 112) in a transactionally consistent manner. The secondarystorage 112 provides methods for inserting, deleting, and retrieving thesecond records 116. Moreover, the secondary storage 112 providesdurability (e.g., data is not lost upon occurrence of a crash).Optionally, the secondary storage 112 can include one or more indexes(e.g., the indexes can enhance speed of performing queries). Accordingto various examples, the secondary storage 112 can be implemented as adatabase management system (DBMS) table, a key-value store, a file, orthe like.

The first records 114 and the second records 116 are separated, with thememory 104 including the first records 114 and the data store 106including the second records 116. According to an example, the memory104 can include one or more access filters (e.g., a Bloom filter, arange filter, a combination thereof, etc.). The access filter(s) caninclude a summary of contents of at least a portion of the secondarystorage 112. The storage interface component 120 can check the accessfilter(s) to identify whether to access the secondary storage 112. Otherthan the access filter(s), the memory 104 need not include informationabout the second records 116, such as keys in in-memory indexes.Further, the storage interface component 120 can enable a transaction(e.g., a unit of work) processed by the database management system 108against the database to access and update the records 114-116 in theprimary storage 110 and the secondary storage 112.

The database management system 108 can support a multi-versionconcurrency control (MVCC) scheme. As part of such scheme, records aremulti-versioned and versions of a record have disjoint valid timeranges. Transactions read records as of a logical read time, while anupdate of a record creates a new version of the record (e.g.,multi-versioning). By way of example, the storage interface component120 can perform a read by checking the primary storage 110 in the memory104 for a key; if the key is not found in the primary storage 110 or alookup index is not unique, then the storage interface component 120 candetermine whether to access the secondary storage 112 (e.g., using anaccess filter). Upon determining to access the secondary storage 112,the storage interface component 120 can perform a read of the secondarystorage 112.

Moreover, the memory 104 includes a private cache 122. The private cache122 can retain a record read by the storage interface component 120 fromthe secondary storage 112 for a transaction (e.g., a copy of the recordas stored in the secondary storage 112). For example, the storageinterface component 120 can read a record from the secondary storage 112and store the record in the private cache 122. For simplicity, oneprivate cache (the private cache 122) for a particular transaction isshown in FIG. 1; yet, it is contemplated that each transaction processedby the database management system 108 can have a respective privatecache to store records read from the secondary storage 112. Otherprivate caches can be substantially similar to the private cache 122.

Since the second records 116 in the secondary storage 112 may beaccessed infrequently, the private cache 122 can be used to store arecord (e.g., one of the second records 116) read from the secondarystorage 112 for a given transaction due to a low probability of anothertransaction reading the same record within a time window (e.g., timewindow during which the private cache 122 retains the record). If agiven record (e.g., one of the second records 116) from the secondarystorage 112 is read twice by the storage interface component 120 for twodifferent transactions, then two versions of the given record can beretained in private caches (e.g., one cached version for each of thetransactions).

The storage interface component 120 maintains transactional consistencyfor updates that span the primary storage 110 and the secondary storage112 (e.g., when data moves from primary storage 110 to secondary storage112 or vice versa) even if the secondary storage 112 is nottransactional. More particularly, the storage interface component 120can include an update memo 124, which can be used to maintaintransactional consistency (e.g., the memory 104 can include the updatememo 124). The update memo 124 can be a table (e.g., a durable table)retained in the memory 104 that temporarily stores informationconcerning records having updates that span the primary storage 110 andthe secondary storage 112.

The update memo 124 can temporarily store timestamp notices that specifystatuses of at least a subset of the second records 116 in the secondarystorage 112. By using the update memo 124, the secondary storage 112need not be transactional. The storage interface component 120 canaccess a record of the database for a transaction based on at least oneof the timestamp notices of the update memo 124. Moreover, the storageinterface component 120 can use the update memo 124 to perform livemigration of data to and from the secondary storage 112 while thedatabase is online and active. Further, since the memory 104 includesthe update memo 124, timestamp updates can be performed by the storageinterface component 120 during post-processing of a transaction; thetimestamp updates during transaction post-processing can be performed inthe memory 104 without accessing the secondary storage 112. The storageinterface component 120 can also perform validation in the memory 104without accessing the secondary storage 112. By performing transactionpost-processing and validation in the memory 112 (without accessing thesecondary storage 112), a number of accesses of the secondary storage112 by the storage interface component 120 can be minimized. Forexample, the secondary storage 112 can be accessed by the storageinterface component 120 to read, insert, or delete the second records116; yet, the secondary storage 112 need not be accessed for transactionpost-processing or validation.

Turning to FIG. 2, illustrated is a system 200 that accesses andmigrates the records 114-116 retained in the primary storage 110 and thesecondary storage 112. The system 200 includes the storage interfacecomponent 120, the primary storage 110, and the secondary storage 112.As noted above, the primary storage 110 stores the first records 114 andthe secondary storage 112 stores the second records 116. The primarystorage 110 retains records (e.g., the first records 114) that areaccessed frequently, while the secondary storage 112 retains lessfrequently accessed records (e.g., the second records 116). Forinstance, the secondary storage 112 can retain archival data that israrely accessed but part of the database with which the databasemanagement system 108 interacts.

The storage interface component 120 provides a unified interface foraccessing the records 114-116 of the database, which are distributedbetween the primary storage 110 and the secondary storage 112. Thestorage interface component 120 provides an interface to upper layers ofthe database management system 108 (e.g., layers above the storageinterface component 120) and applications that interact with thedatabase management system 108, while obscuring a physical location of arecord from the upper layers of the database management system 108 aswell as the applications that interact with the database managementsystem 108. Thus, the storage interface component 120 can obscure asplit of the records 114-116 of the database between the primary storage110 and the secondary storage 112. Moreover, utilization of the storageinterface component 120 can enable decreasing an amount of overheadcaused by accessing the secondary storage 112.

The storage interface component 120 can automatically and transparentlymove records between the primary storage 110 and the secondary storage112 based on access patterns for the records. The storage interfacecomponent 120 can migrate the records between the primary storage 110and the secondary storage 112 while the database management system 108is online and operating (e.g., migration can be performed withoutaffecting active transactions). To migrate records between the primarystorage 110 and the secondary storage 112, the storage interfacecomponent 120 can perform a migration using operations (insert anddelete) wrapped in a transaction. This allows the migration transactionto run concurrently with other transaction(s), with the migrationtransaction and the other transaction(s) being isolated and consistencyof the database being maintained.

The storage interface component 120 can enable a transaction 202, whichcan be performed in the database management system 108 against thedatabase, to access and update the records 114-116 in the primarystorage 110 and the secondary storage 112. The transaction 202 caninclude a set of operations and/or queries that can be processed by thedatabase management system 108. The split of the records 114-116 betweenthe primary storage 110 and the secondary storage 112 is transparent tothe transaction 202. Moreover, the private cache 122 corresponds to thetransaction 202; the private cache 122 stores one or more records (e.g.,from the second records 116) read as part of the transaction 202 by thestorage interface component 120 from the secondary storage 112. A memoryheap (e.g., the private cache 122), separate from memory heaps of othertransactions, is reserved for the transaction 202. Cached records forthe transaction 202 can be stored in the memory heap. Further, when thetransaction 202 terminates, memory reserved by the heap is released.

The storage interface component 120 includes a control component 204that manages performance of an operation upon a record (or more than oneof the records 114-116), where the transaction 202 specifies theoperation. One or more operations can be wrapped in the transaction 202.Moreover, the control component 204 can enable traversal over therecords 114-116 in the primary storage 110 and the secondary storage 112in connection with performance of the operation (or operations).

The storage interface component 120 further includes a primary indexscanner component 206 that scans an index (or indexes) of the primarystorage 110. The primary storage 110 includes one or more indexes. Theprimary index scanner component 206 can perform a lookup based on theindex(es) of the primary storage 110 to enable the control component 204to access, insert, and delete records in the primary storage 110.

The primary storage 110 supports various types of indexes that can bescanned by the primary index scanner component. Examples of types ofindexes include hash indexes, ordered indexes, and so forth; yet, it iscontemplated that other types of indexes can additionally oralternatively be supported by the primary storage 110. By way ofillustration, hash indexes can be implemented using lock-free hashtables. Pursuant to another illustration, ordered indexes can beimplemented using tree data structures, such as a B-trees, Bw-trees(e.g., lock-free versions of B-trees), and so forth.

Moreover, the storage interface component 120 includes a secondary indexscanner component 208 that enables the control component 204 to access,insert, and delete records in the secondary storage 112. As noted above,the secondary storage 112 may optionally include one or more indexes. Ifthe secondary storage 112 includes such index(es), the secondary indexscanner component 208 can scan the index(es) of the secondary storage112. According to an example, the secondary storage 112 may include thesame types of index(es) as the primary storage 110. Following thisexample, the secondary index scanner component 208 can likewise performa lookup based on the index(es) of the secondary storage 112. Yet, it isto be appreciated that the claimed subject matter is not limited to theforegoing example (e.g., the secondary storage 112 may lack an indexwhile providing other methods for the secondary index scanner component208 to employ for inserting, accessing, and deleting records).

The storage interface component 120 may optionally include an accessfilter 210. While FIG. 2 shows one access filter (e.g., the accessfilter 210), it is contemplated that the storage interface component 120can include more than one access filter. The access filter 210 can be atable retained in the memory 104. The access filter 210 stores a compactsummary of contents of the secondary storage 112 (or a portion of thecontents of the secondary storage 112). Pursuant to an example, indexesof the secondary storage 112 can be associated with corresponding accessfilters; yet, as noted above, according to other examples, the secondarystorage 112 may lack indexes.

Prior to accessing the secondary storage 112, the control component 204checks the access filter 210 (or a plurality of access filters) toidentify whether to access the secondary storage 112. If the accessfilter 210 indicates that no records in the secondary storage 112satisfy search criteria, then the control component 204 can skipaccessing the secondary storage 112, thereby avoiding an unnecessaryaccess of the secondary storage 112.

Examples of types of the access filter 210 include a Bloom filter, arange filter, and so forth. A Bloom filter, for instance, can be used bythe control component 204 for point lookups and a range filter can beused by the control component 204 for range queries. By way ofillustration, the storage interface component 120 can include aplurality of differing types of access filters. Additionally oralternatively, the storage interface component 120 can include aplurality of a particular type of access filter.

According to an example, the storage interface component 120 can includea Bloom filter for a hash index of the secondary storage 112 (e.g., oneBloom filter per hash index). By way of another example, the storageinterface component 120 can include a set of Bloom filters for a hashindex of the secondary storage 112; the Bloom filters in the set caneach cover records in adjacent hash buckets. Following this example,each Bloom filter in the set can be smaller than a Bloom filter for thehash index. Use of the plurality of smaller Bloom filters as opposed toa single larger Bloom filter for the hash index can result in thefollowing: (a) the smaller Bloom filters can be built faster; (b) filterlookups performed by the control component 204 can be faster (e.g.,fewer cache misses); (c) access skew can be exploited by assigning fewerbits to infrequently used Bloom filters as compared to more frequentlyaccessed Bloom filters; and (d) Bloom filters that have deterioratedbecause of inserts and deletes can be rebuilt by scanning the hashbuckets.

When the control component 204 receives a record from the secondaryindex scanner component 208 (e.g., one or more of the second records 116from the secondary storage 112), the control component 204 can checkwhether the record is visible. Moreover, the control component 204 canevaluate whether the record satisfies a filter predicate (if any). Upondetermining that the record qualifies, the control component 204 copiesthe record to the private cache 122. A copy of a record stored in theprivate cache 122 is referred to herein as a cached record. The controlcomponent 204 can set a first link field of the cached record to a value“COLD” and clear other link fields of the cached record. The controlcomponent 204 further sets a notice pointer (NoticePtr) field in aprefix of the cached record to include a pointer to a timestamp noticein the update memo 124 if a timestamp notice related to the cachedrecord exists in the update memo 124. Further, the control component 204returns a pointer to a portion of the cached record without the prefix(a pointer to a real record part of the cached record as describedbelow).

Whether the primary storage 110 or the private cache 122 retains arecord can be transparent to upper layers of the database managementsystem 108 that interface with the storage interface component 120 (andapplications that interact with the database management system 108).However, it is contemplated that the control component 204 can set afirst link field of a cached record in the private cache 122 to “COLD”to enable identifying which records originated from the secondarystorage 112 during validation.

As noted above, the update memo 124 can be a table (e.g., a durabletable) retained in the memory 104 that temporarily stores timestampnotices that specify statuses of at least a subset of the second records116 in the secondary storage 112. The storage interface component 120,for example, can include one update memo (e.g., the update memo 124) perdatabase. The update memo 124 enables validation and detection ofwrite-write conflicts to be performed in memory without accessing thesecondary storage 112.

The update memo 124 is used by the control component 204 when reading arecord from the secondary storage 112 and during validation. Whenreading a record, the control component 204 checks whether the updatememo 124 includes a matching timestamp notice for the record. If theupdate memo 124 includes a matching timestamp notice for the record,then the timestamp notice specifies a current status and timestamps forthe record.

According to an example, an update transaction (e.g., the transaction202), running under repeatable read or serializable isolation, validatesits reads by rechecking the timestamps of records that are read. Theupdate memo 124 stores the current timestamps of records updated since abegin time of an oldest active transaction. Thus, the update transactioncan validate reads by checking timestamp notices in the update memo 124.

FIGS. 3-4 illustrate an example of storing, accessing, and updatingrecords without separating the records between the primary storage 110and the secondary storage 112. Rather, in the example described in FIGS.3-4, the primary storage 110 retains the records. It is to beappreciated that the example of FIGS. 3-4 is provided for illustrationpurposes, and the claimed subject matter is not so limited.

FIG. 3 depicts an exemplary structure 300 of a record that can be storedin the primary storage 110. The structure 300 includes a header, anumber of link (pointer) fields, and a payload. The header includes abegin timestamp field (BeginTs) and an end timestamp field (EndTs). Thedatabase management system 108 uses multi-versioning, where an update toa record creates a new version of the record; thus, a valid time periodof a version of a record is set by a begin timestamp and an endtimestamp respectively stored in the begin timestamp field and the endtimestamp field of the header.

Moreover, a number, type, etc. of fields in the payload can be defined(e.g., user defined, etc.). In the example illustrated in FIGS. 3-4, thepayload includes three fields (for the exemplary bank account tableillustration): Name, City, and Amount. Again, it is to be appreciatedthat the claimed subject matter is not limit to this example, which isprovide for illustration purposes.

FIG. 4 shows an exemplary table 400 that includes five version records.While not shown, it is contemplated that the table 400 can includesubstantially any number of additional version records.

As shown in FIG. 4, the exemplary table 400 has two indexes; a hashindex 402 on the Name field and an ordered index 404 on the City field.Each index can use a respective link field in the records. As depicted,a first link field is reserved for the hash index 402 and the secondlink field is reserved for the ordered index 404.

For illustration purposes, a hash function used for the hash index 402can select a first letter of a name in the Name field. Version recordsthat hash to the same bucket are linked together using the first linkfield (represented by arrows 406-410). Moreover, leaf nodes of a treedata structure (e.g., a B-tree, a Bw-tree, etc.) can store pointers torecords. If multiple records have the same key value, duplicates arelinked together using the second link field (represented by arrows412-414) in the records and the tree data structure can point to arecord on the chain (e.g., first record on the chain, etc.).

Hash bucket J includes three records: two versions of a record for Johnand one version of a record for Jane. Jane's single version record(Jane, Paris, 150) has a valid time period from time 15 to infinity,meaning that such version record was created by a transaction thatcommitted at time 15 and is still valid. John's older version record(John, London, 100) was valid from time 10 to time 20; at time 20,John's older version record was updated. The update created a newversion record (John, London, 110), which has a valid time period fromtime 20 to infinity.

Various operations can be performed on the version records of theexemplary table 400. The operations can include reading a record,updating a record, deleting a record, and inserting a record.

A read operation specifies a logical (as-of) read time and only versionswhose valid time periods overlap the read time are visible to the read;other versions are ignored. Different versions of a record havenon-overlapping valid time periods, so at most one version of a recordis visible to a read at a given time. A lookup for John with read time15, for example, can trigger a scan of bucket J that checks records inthe bucket. The scan returns the record with Name equal to John andvalid time period 10 to 20. If the index on Name is declared to beunique, the scan of the buckets can stop upon finding a qualifyingrecord.

According to another example, an update operation can be performed.Bucket L includes two versions of a record that belong to Larry.Transaction 75 is in the process of deducting $20 from Larry's account.As part of the update, a new record (Larry, Rome, 150) is created andinserted into the index.

Note that a transaction Id (Tx75) of transaction 75 is stored in theBeginTs and EndTs fields of the new and old version, respectively (e.g.,a bit in the BeginTs field can indicate content type of the BeginTsfield, a bit in the EndTs field can indicate content type of the EndTsfield). A transaction Id stored in the EndTs field can inhibit othertransactions from updating the same version of the record. When read, atransaction Id stored in the BeginTs field indicates that the versionmay not yet be committed and identifies the transaction that created theversion.

Now suppose transaction 75 commits with end timestamp 100. Aftercommitting, transaction 75 returns to the old and new versions and setsthe BeginTs and EndTs fields to 100. The final values are shown in FIG.4 by the shading below the old and new versions. The old version (Larry,Rome, 170) now has the valid time period 30 to 100 and the new version(Larry, Rome, 150) has a valid time period from 100 to infinity.

The foregoing example of an update operation also illustrates delete andinsert operations for records in the primary storage 110. An updateoperation can include deleting an existing version and inserting a newversion. Moreover, obsolete versions can be discarded; for instance, aversion of a record can be discarded when no longer visible to anyactive transaction. Cleaning out obsolete versions (e.g., garbagecollection), for instance, can be handled cooperatively by workerthreads.

Again, reference is made to FIG. 1. The database management system 108can utilize optimistic MVCC to provide snapshot, repeatable read, andserializable transaction isolation without locking Below are variousfeatures of optimistic MVCC that can be implemented in the databasemanagement system 108.

The database management system 108 can use timestamps produced by amonotonically increasing counter. Below are various uses of thetimestamps by the database management system 108.

The timestamps can be used as a commit time (e.g., end time) of atransaction. An update transaction commits at a distinct point in timecalled the commit or end timestamp of the transaction. The commit timedetermines a position of the transaction in a transaction serializationorder.

Moreover, the timestamps can be utilized to specify a valid time periodfor a version of a record. For instance, each record in the database caninclude two timestamps, namely, a begin timestamp and an end timestamp.The valid time period for a version of a record is a timestamp range setby the begin timestamp and end timestamp.

The timestamps can further be used for a logical read time. Moreparticularly, a read time of a transaction is set to a start time of thetransaction. For instance, only versions of records with valid timeperiods that overlap the logical read time of a transaction are visibleto the transaction.

Version visibility supports concurrency control in the databasemanagement system 108. For example, a particular transaction can executewith logical read time. Following this example, versions of records withboth begin timestamps less than the logical read time and end timestampsgreater than the logical read time are visible to the particulartransaction; other versions of records (other than updates for theparticular transaction which are visible to the particular transaction)are not visible to the particular transaction. Operation(s) of atransaction can be performed on versions of records that are visible tothe transaction, while the operation(s) are unable to be performed onversions of records that are not visible to the transaction.

The following generally describes transaction commit processing. Uponcompletion of normal processing of a transaction by the databasemanagement system 108, the database management system 108 can beingcommit processing of the transaction. Commit processing, for example,can include validation, logging, and post-processing.

Validation is performed for update transactions running at repeatableread or serializable isolation; validation need not be performed forread-only transactions or update transactions at lower isolation levels.Validation begins by obtaining an end timestamp of a transaction. Then,reads performed as part of the transaction are verified and, if executedunder serializable isolation, the storage interface component 120verifies that no phantoms have appeared.

To validate reads for the transaction, the storage interface component120 checks that the versions that have been read are visible as of theend time of the transaction (e.g., specified by the end timestamp). Tocheck for phantoms, index scans are repeated to look for versions thathave become visible since the transaction began (e.g., specified by thebegin timestamp). To enable validation, the storage interface component120 can maintain a read set and a scan set for each transaction. A readset for a transaction includes pointers to versions of records read aspart of the transaction. A scan set includes information used to repeatscans.

Moreover, logging and post-processing can be performed by the storageinterface component 120. For instance, a transaction T can be committedupon updates to the database being written to a transaction log.Transaction T writes to the log the contents of new versions created byT and the primary key of versions deleted by T.

Upon the updates of transaction T being logged, T is committed (e.g.,irreversibly). Thereafter, the storage interface component 120 can begina post-processing phase for transaction T during which the begin and endtimestamps in versions affected by the transaction are updated toinclude the end timestamp of the transaction T. For instance, a writeset can be maintained for a transaction; the write set can includepointers to inserted and deleted versions of records, which can be usedto perform the timestamp updates and generate the log content.

Turning to FIG. 5, illustrated is an exemplary structure 500 of a record(e.g., one of the second records 116) in the secondary storage 112. Thestructure 500 includes a transaction Id (TxnId) field, a key field, anda payload. The key field and the payload correspond to counterparts in astructure of a record in the primary storage 110. Further, the TxnIdfield of a record stores an Id of a transaction (e.g., a migrationtransaction) that inserted such record into the secondary storage 112.TxnId can serve as a version number used to identify different versionsof a common record.

Now referring to FIG. 6, illustrated is an exemplary structure 600 of acached record in the private cache 122. The structure includes a realrecord portion 602, which can include a structure that matches astructure of a record in the primary storage 110. For instance, the realrecord portion 602 can include a BeginTs field, an EndTs field, linkfields (e.g., with a first link field including a value set to COLD), akey field, and a payload. Moreover, a prefix 604 of the structure 600includes a notice pointer (NoticePtr) field. The NoticePtr field caninclude a pointer to a timestamp notice in the update memo 124 or null.Such pointer can be used during validation.

Turning to FIG. 7, illustrated is an exemplary structure 700 of atimestamp notice in the update memo 124. Similar to the other structuresdescribed herein, the structure 700 includes a BeginTs field and anEndTs field. According to the depicted example, a header of thestructure 700 can also include other miscellaneous fields; yet, theclaimed subject matter is not so limited.

A payload of the structure 700 can include various fields. Moreparticularly, the payload can include fields to identify a target recordof a notice: TableId, RecTxnId, and Key. The TableId field can includean Id of a table to which the target record belongs. The RecTxnId fieldcan include a value from the TxnId field of the target record. The Keyfield can be a variable length field storing a key of the target record.The secondary storage 112 can include multiple versions of a commonrecord, but the versions have different TxnId values. Moreover, aBTsCopy field can be used for storing a copy of a timestamp.

A timestamp notice includes timestamps for a target record; thetimestamp notice is used when the record is read into a private cache(e.g., the private cache 122). The structure 700 can further include aType field. The Type field can include a value that specifies a type oftimestamps carried by a notice. Examples of the types are set forthbelow:

-   -   N (None)—temporary notice with no timestamps. The target record        was orphaned because of a failed migration transaction and is to        be ignored.    -   B (Begin)—the begin timestamp of the timestamp notice equals the        begin timestamp of the target record.    -   E (End)—the begin timestamp of the timestamp notice equals the        end timestamp of the target record.    -   BE (Begin & End)—the BTsCopy field includes the begin timestamp        of the target record and the begin timestamp of the timestamp        notice equals the end timestamp of the target record.

Turning to FIG. 8, illustrated is a system 800 that performs variousoperations in connection with use of the secondary storage 112. Thesystem 800 includes the database management system 108, the primarystorage 110, the secondary storage 112, and the private cache 122. Thedatabase management system can process the transaction 202. Thetransaction 202 can include a set of operations and/or queries that canbe conducted by the database management system 108.

As described above, the database management system 108 includes thestorage interface component 120. The storage interface component 120includes the control component 204, the primary index scanner component206, the secondary index scanner component 208, the update memo 124, andthe access filter 210. Moreover, the control component 204 can includean insertion component 802 that inserts new records into the primarystorage 110. Further, a new version of a record resulting from an updateto the record can be inserted into the primary storage 110 (by an updatecomponent 808 as described below); the foregoing can occur regardless ofwhere the old version resides.

The control component 204 also includes a migration component 804.According to an example, the migration component 804 can classifycandidate records for migration to and from the secondary storage 112.The migration component 804 can log record accesses (e.g., a sample ofrecord accesses) to enable estimating access frequencies. Pursuant toanother example, estimation of access frequencies can be performed by adisparate component of the database management system 108 (or separatefrom the database management system 108, the estimation can be performedoffline, etc.); the migration component 804 can receive informationtriggering migration of one or more records from such disparatecomponent.

The migration component 804 can trigger migration to and from secondarystorage 112 using the results of classification. The migration component804 can migrate a record while the database is online and active.Migrating data from the primary storage 110 to the secondary storage 112is now described (an update operation can be used to migrate data fromthe secondary storage 112 to the primary storage 110 as describedlater). Inserts into the secondary storage 112 are performed by a set oftwo migration transactions that move records from the primary storage110 to the secondary storage 112. According to an example, the migrationcomponent 804 can cause migration to be executed in the background whenthe database management system 108 is lightly loaded. Migration of asingle record is discussed below, but such techniques can similarlyapply when multiple records are concurrently migrated.

The migration component 804 migrates a record from the primary storage110 to the secondary storage 112 by deleting the record from the primarystorage 110 and re-inserting a corresponding record (e.g., with a commonkey and payload) in the secondary storage 112. During processing of thefirst migration transaction, the migration component 804 can read arecord from the primary storage 110 and install a timestamp notice inthe update memo 124. The timestamp notice can include an indicator toignore a corresponding record in the secondary storage 112, where thecorresponding record pertains to the record. Moreover, during processingof the second migration transaction, the migration component 804 caninsert the corresponding record in the secondary storage 112, update thetimestamp notice in the update memo 124 to remove the indicator toignore the corresponding record in the secondary storage 112, and deletethe record in the primary storage 110. During migration, the databasemanagement system 108 is still active and other transactions may berunning, some of which may attempt to read or update the record beingmigrated.

The following exemplary algorithm (Algorithm 1) describes migration of arecord from the primary storage 110 to the secondary storage 112. Atleast a portion of Algorithm 1 can be performed by the migrationcomponent 804. It is to be appreciated that the algorithm is providedfor illustration purposes, and the claimed subject matter is not limitedto the example set forth in this algorithm.

Algorithm 1: MigrateRecordToSecondaryStorage 1. Begin transaction.Denote transaction Id by TnN. 2. Read the target record from the primarystorage into a buffer TrgRec. 3. If no record was found, abort thetransaction and report “no record found. 4. Check whether a begintimestamp of the record is less than the read time of the oldest activetransaction and an end timestamp equals infinity. If not, abort thetransaction and report “record can't be migrated.” 5. Create a timestampnotice Ntc, setting fields of Ntc as follows: TableId = Id of targettable, RecTxnId = TxN, Key = TrgRec.Key, Type = N, BTsCopy = 0. 6.Insert Ntc into the update memo. 7. Commit transaction. 8. Begintransaction. Denote transaction Id by TxM. 9. Set TrgRec.TxnId = TxN.10. Insert TrgRec into secondary storage. 11. Update affected Bloomfilters. 12. Update notice Ntc in the update memo, setting Type = B. 13.Delete TrgRec from the primary storage. 14. Commit transaction.

Because the secondary storage 112 need not be transactional, themigration component 804 can perform the migration from the primarystorage 110 to the secondary storage 112 in two steps, each in aseparate transaction. The first migration transaction is referred to asTxN, and the second migration transaction is referred to as TxM. Incontrast, if such migration were performed in a single transaction andthe transaction aborted, it is possible that two valid instances of thesame record may result, one in the primary storage 110 and one in thesecondary storage 112.

During processing of the first transaction (TxN, steps 1-6 of Algorithm1), the migration component 804 reads a target record and installs apreliminary migration timestamp notice Ntc of type N in the update memo124. The timestamp notice provides an indication to ignore the targetrecord as part of processing of other transactions (e.g., transactionsother than the first transaction TxN). A record is not migrated unlessit is a latest version (e.g., end timestamp of infinity) and visible tocurrent and future transactions (checked in step 4). Versioning in thedatabase management system 108 can be transient, where an old version iskept only until no longer visible to any currently active transactions.Moreover, old versions can be cleaned, so such versions need not bemigrated to the secondary storage 112. The first transaction TxN commitsin step 7 of Algorithm 1.

During processing of the second transaction (TxM, steps 8-13 ofAlgorithm 1), the migration component 804 performs the migration. Themigration component 804 creates a copy of the target record, inserts thecopy of the target record into the secondary storage 112, and updatesaffected Bloom filter(s). Once the record has been inserted into thesecondary storage 112, the migration component 804 updates the migrationtimestamp notice in the update memo 124, setting its type to B.Thereafter, the migration component 804 deletes the record in theprimary storage 110 and the transaction commits. The begin timestampstored in the migration timestamp notice matches an end timestamp of theversion in the primary storage 110. The migration timestamp notice inthe update memo 124 is updated in the same transaction as deletion ofthe version in the primary storage 110, which provides that the twoversions have non-overlapping timestamp ranges (e.g., one of the twoversions may be visible to a transaction, but not both). The secondtransaction TxM commits in step 14 of Algorithm 1.

If transaction TxM fails to commit because of a crash or the like,changes (other than possibly the insert into the secondary storage 112)are automatically rolled back by the database management system 108. Ifthe insert into the secondary storage 112 is not rolled back, then thenew version can continue to exist in the secondary storage 112; however,such version is associated with a timestamp notice of type N, whichindicates to ignore the version of the record.

FIG. 9 illustrates an exemplary effect of migrating a record from theprimary storage 110 to the secondary storage 112. At 900, a status aftercommit of the first transaction, TxN, is depicted (e.g., before commitof the second transaction). A commit time (e.g., an end timestamp) ofthe first transaction is CtN. At 902, a status after commit of thesecond transaction, TxM, is depicted. A commit time (e.g., an endtimestamp) of the second transaction is CtM. As depicted at 900, atimestamp notice is inserted into the update memo 124 responsive tocommit of the first transaction TxN. As depicted at 902, the version inthe primary storage 110 is deleted responsive to commit of the secondtransaction TxM; the version in the primary storage 110 is deleted bysetting the end timestamp CtM in the EndTs field. Moreover, responsiveto commit of the second transaction TxM, a copy of the record isinserted into the secondary storage 112 and the timestamp notice isupdated to a type B and to have the end timestamp CtM in the BeginTsfield.

Again, reference is made to FIG. 8. The old version in the primarystorage 110 and the old version of the timestamp notice of type N can begarbage collected upon no longer being visible to an active transaction(e.g., by a cleaner component 812 described herein). The new version ofthe timestamp notice can be deleted (e.g., by the cleaner component 812)as soon as the version in the secondary storage 112 is visible to activetransactions (e.g., when the read time of the oldest active transactionin the system becomes higher than CtM in the example of FIG. 9).

According to an example, it is possible that a transaction T may attemptto read or update a record R in the primary storage 110, where R isunder active migration by the migration component 804. In this case, Rhas an end timestamp of TxM, meaning that R is in the process of beingmoved to the secondary storage 112 and a type of a timestamp notice forR is being changed responsive to processing of transaction TxM (steps7-13 in Algorithm 1). In this case, T follows version visibilitysupported by the database management system 108. If R is read responsiveto processing of T, R is ignored if TxM is active or aborted. Ifpreparing to commit TxM, then R is visible to T if a commit time of TxMis greater than a read time of T; otherwise, T speculatively ignores R(e.g., T takes a commit dependency on TxM). If TxM is committed, then Tuses an end timestamp of TxM to test visibility. T is allowed to updateR only if TxM has aborted; if TxM is active or preparing, then this iswrite-write conflict and T aborts.

Moreover, the control component 204 can include a deletion component 806that can delete a record from the secondary storage 112. Deleting arecord from the secondary storage 112 can be a two-step process. Thedeletion component 806 can first mark the record logically deleted bycreating a timestamp notice of type E or BE in the update memo 124. Thetimestamp notice specifies the end timestamp of the record. The updatememo 124 is checked when reading the corresponding record so thetimestamp notice can be found and checked to identify the visibility ofthe record. If the record is not visible, the record can be ignored whenread. The record can be physically removed later by the cleanercomponent 812. The record can be removed and the timestamp noticedeleted when no longer visible to any active transaction.

The deletion component 806 can read a record for deletion from thesecondary storage 112. Moreover, the deletion component 806 can searchthe update memo 124 for a matching timestamp notice for the record. Thedeletion component 806 can also mark the record logically deleted in theupdate memo 124 responsive to searching for the matching timestampnotice by updating the matching timestamp notice in the update memo 124(if found) or inserting a new timestamp notice in the update memo 124for the record (if not found).

The following exemplary algorithm (Algorithm 2) describes deletion of arecord from the secondary storage 112. Algorithm 2 can be performed, forinstance, at least in part by the deletion component 806. It is to beappreciated that the algorithm is provided for illustration purposes,and the claimed subject matter is not limited to the example set forthin this algorithm.

Algorithm 2: DeleteRecordFromSecondaryStorage  1. Begin transaction.Denote transaction Id by TxD.  2. Read the target record from thesecondary storage into a buffer TrgRec.  3. If no record was found,abort the transaction and report “no record found.”  4. Look for amatching (and visible) timestamp notice in the update memo.  5. If notimestamp notice is found, insert a timestamp notice into the updatememo with TableId = Id of target table, RecTxnId = TrgRec.TxnId, Key =TrgtRec.Key, Type = E, BTsCopy = 0.  6. If a timestamp notice of Type Nif found, abort the transaction and report “no record found.”  7. If atimestamp notice of Type B is found, update the timestamp notice bychanging its Type field to BE and setting the BTsCopy field as follows.7.1. If a begin timestamp field of the timestamp notice includes atransaction ID TxM, look up the transaction object for TxM and copy itscommit timestamp into BTsCopy. It has a commit timestamp because thetimestamp notice is visible. 7.2. If the begin timestamp field of thetimestamp notice includes a timestamp, copy it into BTsCopy.  8. If anotice of type BE or E is found, the record has already been deleted soabort the transaction and report “no record found.“  9. If the record isdeleted because it is being updated, the new version can be insertedinto the primary storage at this point. 10. Commit transaction.

According to an example, a record in the secondary storage 112 may bedeleted because it is being updated. In such a scenario, the new versionis inserted into the primary storage 110 in the same transaction.

FIG. 10 illustrates an exemplary effect of deletion of a record from thesecondary storage 112. At 1000, depicted is the record in the secondarystorage 112 and a timestamp notice (which may or may not exist) beforeTxD commits. The record in the secondary storage 112 remains unchangedafter TxD commits at time CtD. At 1002, various examples of thetimestamp notice after TxD commits at time CtD are depicted. Asillustrated, timestamp notice(s) are added and/or updated in the updatememo 124 as a result of the deletion (e.g., depending upon whether atimestamp notice of Type B for the record was in the update memo priorto commit of TxD). The begin timestamp of a new version of a notice or anew notice after commit at time CtD specifies the end timestamp thetransaction TxD.

Reference is again made to FIG. 8. Write-write conflicts can be detectedin connection with deletion. Suppose two transactions attempt toconcurrently delete or update the same record. A write-write conflict inthe primary storage 110 is detected because each record has one copy inmemory. One of the transactions updates an end timestamp of the recordfirst; when the second transaction attempts to change the end timestampof the record, it is detected that the end timestamp of the record is nolonger equal to infinity. Thus, the second transaction can identify thatanother transaction has or is about to delete the version, and canabort.

With cold records in the secondary storage 112, each transaction reads acopy from disk and obtains a cached version in a buffer corresponding tothe transaction. An update or deletion is performed by inserting orupdating a timestamp notice in the update memo 124. If there is anexisting timestamp notice to be modified, a conflict can be detectedwhen attempting to modify the end timestamp of the notice responsive toprocessing of a transaction. However, there may be no old notice tomodify. According to an example, the update memo 124 can have a uniqueindex built on the fields TableID, RecTxnId, and Key. Following thisexample, two transactions trying to modify the same record version endup trying to insert two timestamp notices in the update memo 124 withthe same key value. The unique index on the update memo can be used todetect the violation, causing and one of the transactions to abort.

Moreover, the control component 204 can include an update component 808that can update a record in the secondary storage 112 by deleting theold version from the secondary storage 112 and inserting a new versioninto the primary storage 110. The new version, for example, may be latermigrated into the secondary storage 112 (e.g., by the migrationcomponent 804); however, the update component 808 inhibits new versionsof records being directly inserted into the secondary storage 112.Moreover, as discussed above in connection with the insertion component802, new records are similarly inserted into the primary storage 110.

New records and new version of records are inserted into the primarystorage 110 as opposed to the secondary storage 112 since a new recordcan be considered to be hot and updating a record (creating a newversion of a record) can be interpreted as a signal that the record is(potentially) becoming hot. Inserting the new records and the newversions in the primary storage 110 allows a scan validation, performedfor serializable transactions, to be executed in the primary storage 110(e.g., in the memory 104). Scan validation checks for new recordssatisfying a scan predicate that have appeared since a transactionbegan. By inserting modified and new records in the primary storage 110,the secondary storage 112 need not be checked as part of the scanvalidation.

The control component 204 can further include a read component 810 thatcan read a record from the secondary storage 112 (as part of atransaction). The read component 810 can determine whether to access thesecondary storage 112 for the record using the access filter 210. Theread component 810 can use a point lookup or scan to probe the in-memoryaccess filter 210 to determine whether to access the secondary storage112. If the access filter 210 specifies to access the secondary storage112 based upon the probe, then the read component 810 can read therecord. Responsive to determining to access the secondary storage 112for the record, the read component 810 can retrieve the record from thesecondary storage 112. The read component 810 can further search theupdate memo 124 for a matching timestamp notice for the record.Timestamps for the record can be set by the read component 810 basedupon whether the matching timestamp notice is found and data indicatedby the matching timestamp notice (if found). The read component 810 canfurther insert the record with the timestamps into the private cache 122for the transaction.

The below exemplary algorithm (Algorithm 3) can be employed at least inpart by the read component 810 to read the record from the secondarystorage 112; yet, the claimed subject matter is not limited to theexample set forth in this algorithm, which is provided for illustrationpurposes.

Algorithm 3: ReadColdRecord  1. Read the target record from secondarystorage into an I/O buffer.  2. Pad the record to the format ofin-memory records (e.g., determine a correct position of the timestampfields before the record in the buffer).  3. Let B and E be localvariables. Set B = 1 and E = infinity.  4. Look for a matching (andvisible) notice in the update memo (using the TxnId and Key fields fromthe record).  5. If a timestamp notice Ntc is found in the update memothen: 5.1. If Ntc is of type N, return a null pointer. 5.2. If Ntc is oftype B, set B = Ntc.Begin, E = infinity. 5.3. If Ntc is of type BE, setB = Ntc.BTsCopy, E = Ntc.Begin. 5.4. If Ntc is of type E, set B = 1, E =Ntc.Begin.  6. Endif  7. Copy B and E into the timestamp fields in theI/O buffer. Set the first link field to COLD.  8. Check whether therecord in the buffer is visible and satisfied user- defined filterfunctions.  9. If the record does not qualify, return Null. 10. Copy therecord from the buffer into a private cache for the transaction. Set aNoticePtr for the record to point to Ntc in the update memo. 11. Returna pointer to the cached copy.

In step 1 of Algorithm 3, the read component 810 can read the recordfrom the secondary storage 112 into an I/O buffer. In step 2, the readcomponent 810 can pad the record so the record has a form that issimilar to a format of in-memory records. At step 4, the read component810 can look for a matching timestamp notice in the update memo 124. Ifa timestamp notice of type N is found (step 5.1), the record is ignored.Otherwise, the begin timestamp and end timestamp of the timestamp noticeare set (depending upon the type of the notice, steps 5.2-5.4). At step8, the read component 810 checks whether the record is visible andpasses user-defined filter function(s). If the record is visible andpasses the filter function(s), the record is copied by the readcomponent 810 into the private cache 122 for the transaction and apointer to the cached record is returned to the transaction.

The control component 204 can further include a cleaner component 812that can clean the secondary storage 112 and/or the update memo 124.Migrating records to the secondary storage 112 and updating or deletingrecords in the secondary storage 112 adds timestamp notices to theupdate memo 124. Records that are deleted from the secondary storage 112(e.g., performed by the deletion component 806) may not be physicallyremoved from the secondary storage 112 responsive to the deletion.Accordingly, the cleaner component 812 can remove stale timestampnotices from the update memo 124. Similarly, the cleaner component 812can remove records in the secondary storage 112 that are no longervisible to any transactions.

The cleaner component 812, for example, can periodically clean timestampnotices no longer visible to active transactions from the update memo124. While the cleaner component 812 performs such cleaning, themigration component 804 can block migration of records. ExemplaryAlgorithm 4, set forth below, provides an example of an algorithm thatcan be performed by at least in part by the cleaner component 812 toclean the update memo 124; again, it is to be appreciated that theclaimed subject matter is not limited to the following exemplaryalgorithm.

Algorithm 4: CleanUpdateMemo   1. Set TSBound to the begin timestamp ofthe oldest active (uncommitted) transaction in the system. 2. For eachcommitted notice Ntc in the update memo do 2.1. DelFlag = false 2.2. //Remove records visible to nobody 2.3. If Ntc is of type N or ((of typeBE or E) and Ntc.Begin < TSBound) then 2.3.1. Delete the record coveredby Ntc from the secondary storage. 2.3.2. DelFlag = true 2.4. Endif 2.5.If Ntc is of type B and Ntc.Begin < TSBound then DelFlag = true 2.6. //Delete the notice 2.7. If DelFlag = true then 2.7.1 Begin transaction2.7.2. Delete Ntc from the update memo 2.7.3. Commit transaction 2.8.Endif

The cleaner component 812 can scan through the update memo 124 to checkcommitted notices. As set forth above, the action taken depends on thetype of the notice. An N type notice indicates a failed insert into thesecondary storage 112. The associated record is not visible for anytransaction, such record can be removed, and the notice can be deleted.

A notice of type BE or E corresponds to a (logically) deleted records.If the record was deleted before the begin timestamp of the oldestactive transaction (TSBound), such record is not visible to current orfuture transactions. Accordingly, the cleaner component 812 can removesuch record from the secondary storage 112 and delete the notice fromthe update memo 124.

A notice of type B corresponds to a currently visible record (e.g., endtimestamp is infinity). If a begin timestamp of the corresponding noticeis earlier than the begin timestamp of the oldest active transaction(TSBound), the record is visible to all current and future transactions.If so, the cleaner component 812 can delete the notice. In thisscenario, the record remains in the secondary storage 112 with nonotice. A record with no notice gets default timestamps (1, infinity)when later read by the read component 810).

The deletion of a notice can be performed by the cleaner component 812in a transaction that begins after the corresponding cold record, ifany, has been deleted (e.g., delete flags can be set, step 2.7 ofAlgorithm 4 can delete notice(s) in a subsequent transaction). By usingseparate transactions, a situation where a notice is deleted but acorresponding record is not can be mitigated.

Moreover, the control component 204 can include a validation component814. An update transaction running at repeatable read or serializableisolation can be validated before it commits. If validation fails, thetransaction aborts.

For both repeatable read and serializable isolation, the validationcomponent 814 can verify that records read as part of execution of thetransaction have not been updated or deleted by another transactionbefore commit. In a memory-only scenario, such verification can beperformed by keeping pointers to record reads. Under such scenario, atcommit time, a test can be performed to verify that the read versionsremain visible as of the commit time. If the versions are still visible,then other transaction(s) have not changed the versions during thelifetime of the validating transaction.

With cold records, more than one copy of the record can be stored.Moreover, as described herein, updates to cold records are performedusing the update memo 124. Accordingly, the validation component 814 canemploy a similar test as compared to the memory-only scenario, where thevalidation component 814 can adjust the end timestamp of the records inthe private cache 122 of the transaction before performing thevisibility check. The validation component 814 can update end timestampsof cached records after processing (e.g., normal processing) of thetransaction and prior to a beginning of validation of a transaction,with the end timestamps being updated based upon timestamp notices thatrespectively correspond to the cached records from the update memo 124.

The following exemplary algorithm (Algorithm 5) describes updating theend timestamp of a cached record Rc, which can be performed by thevalidation component 814. Recall that Rc is prefixed with a NoticePtrfield that is either null or points to the notice used when the recordwas read. Again, it is contemplated that Algorithm 5 is provided forillustration purposes, and the claimed subject matter is not limited tosuch algorithm.

Algorithm 5: PrepareValidationTimestamp 1. Let Rc be a pointer to arecord in the private cache of a transaction. 2. If Rc.NoticePtr is notnull, copy into an end timestamp of Rc one of the following: 2.1. If thenotice is of type E, copy from a begin timestamp of the notice. 2.2. Ifthe notice is of type BE, copy a value from the BTsCopy field of thenotice.

For a serializable update transaction, the validation component 814 canvalidate scans of a transaction to detect phantoms. In a memory-onlyscenario, validation can be performed by repeating the scan against themain-memory index and checking whether new records have appeared.Repeating a scan in the secondary storage 112 can be expensive.Accordingly, repeating scans against the secondary storage 112 can bemitigated by the storage interface component 120 since newly insertedrecords and new versions of records resulting from an update of a coldrecord can be directly inserted into the primary storage 110 (ratherthan the secondary storage 112). Accordingly, a scan of the primarystorage 110 can be employed to detect phantoms.

However, a serializable transaction may fail in various scenario.Examples of such scenarios can include: 1) TxS (Serializable) scans thetable, where TxS is still active; 2) TxI (Insert) inserts a new recordto the primary storage 110 and commits; 3) TxM (Migrate) migrates thenewly inserted record to the secondary storage 112 and commits; and 4)TxS validates by repeating the scan over the primary storage 110, wherethe newly inserted record may not be visible because the version in theprimary storage 110 has been migrated.

To mitigate the foregoing, the validation component 814 can enforce anadditional constraint on migrating records. When the migrationtransaction starts, the validation component 814 can compute TsBoundSer,the begin timestamp of the oldest serializable transaction that isactive (uncommitted) in the system. A migration transaction may notmigrate a record with a begin timestamp later than TsBoundSer.Accordingly, a newly inserted record in the primary storage 110 canremain until the serializable transaction validates.

In general, the storage interface component 120 can reduce a number ofaccesses to the secondary storage 112 (e.g., accesses of the secondarystorage 112 may be slower than accesses of the primary storage 110).According to an example, within the database management system 108, thestorage interface component 120 (e.g., the read component 810) can reada cold record from the secondary storage 112 to bring the record intomemory (the private cache 122 of the transaction 202); thereafter,processing performed on the record can be performed in memory. By way ofother examples, an update or delete of a cold record by the storageinterface component 120 can include (a) a read from the secondarystorage 112 to bring the record into memory (also to verify itsexistence) and (b) a delete to remove the record from secondary storage112 (in case of an update the new version is placed in the primarystorage 110 by the update component 808). The migration of a record fromthe primary storage 110 to the secondary storage 112 can include aninsert into the secondary storage 112. Insert operations (e.g.,performed by the insertion component 802) place a new record in theprimary storage 110, thus do not incur an access to the secondarystorage 112. Moreover, the update memo 124 and the private cache 122 canenable validation and post-processing to occur in memory.

FIGS. 11-12 illustrate exemplary methodologies relating to performingone or more operations on records of a database (e.g., an in-memorydatabase). While the methodologies are shown and described as being aseries of acts that are performed in a sequence, it is to be understoodand appreciated that the methodologies are not limited by the order ofthe sequence. For example, some acts can occur in a different order thanwhat is described herein. In addition, an act can occur concurrentlywith another act. Further, in some instances, not all acts may berequired to implement a methodology described herein.

Moreover, the acts described herein may be computer-executableinstructions that can be implemented by one or more processors and/orstored on a computer-readable medium or media. The computer-executableinstructions can include a routine, a sub-routine, programs, a thread ofexecution, and/or the like. Still further, results of acts of themethodologies can be stored in a computer-readable medium, displayed ona display device, and/or the like.

FIG. 11 illustrates a methodology 1100 of migrating a record from aprimary storage to a secondary storage. Migration of the record can usea first migration transaction and a second migration transaction. Acts1102 and 1104 can be performed during processing of the first migrationtransaction. At 1102, a record can be read from the primary storage. At1104, a timestamp notice can be installed in an update memo. Memory caninclude the update memo. Moreover, the timestamp notice can include anindicator to ignore a corresponding record in the secondary storage,where the corresponding record pertains to the first record (e.g., acopy of at least a portion of the first record).

Further, acts 1106, 1108, and 1110 can be performed during processing ofthe second migration transaction. At 1106, the corresponding record canbe inserted in the secondary storage. At 1108, the timestamp notice inthe update memo can be updated to remove the indicator to ignore thecorresponding record in the secondary storage. At 1110, the first recordcan be deleted in the primary storage.

Turning to FIG. 12, illustrated is a methodology 1200 of reading arecord of a database as part of processing a transaction. At 1202, adetermination can be made concerning whether to access a secondarystorage for the record. The determination can be performed using anaccess filter, which can be retained in memory. At 1204, responsive todetermining to access the secondary storage for the record, the recordcan be retrieved from the secondary storage. At 1206, an update memo canbe searched for a matching timestamp notice for the record. The memorycan include the update memo, and the update memo can include timestampnotices that specify statuses of records of the database having updatesthat span the primary storage and the secondary storage. At 1208,timestamps for the record can be set based upon whether the matchingtimestamp notice is found and data indicated by the matching timestampnotice, if found. At 1210, the record with the timestamps can beinserted into a private cache for the transaction.

Referring now to FIG. 13, a high-level illustration of an exemplarycomputing device 1300 that can be used in accordance with the systemsand methodologies disclosed herein is illustrated. For instance, thecomputing device 1300 may be used in a system that accesses records of adatabase that are split between primary storage and secondary storage.The computing device 1300 includes at least one processor 1302 thatexecutes instructions that are stored in a memory 1304. The instructionsmay be, for instance, instructions for implementing functionalitydescribed as being carried out by one or more components discussed aboveor instructions for implementing one or more of the methods describedabove. The processor 1302 may access the memory 1304 by way of a systembus 1306. In addition to storing executable instructions, the memory1304 may also store an update memo (e.g., timestamp notices), primarystorage (e.g., a first subset of records of a database retained as partof the primary storage), private cache(s), access filter(s), and soforth.

The computing device 1300 additionally includes a data store 1308 thatis accessible by the processor 1302 by way of the system bus 1306. Thedata store 1308 may include executable instructions, a secondary storage(e.g., a second subset of the records of the database retained as partof the secondary storage), etc. The computing device 1300 also includesan input interface 1310 that allows external devices to communicate withthe computing device 1300. For instance, the input interface 1310 may beused to receive instructions from an external computer device, from auser, etc. The computing device 1300 also includes an output interface1312 that interfaces the computing device 1300 with one or more externaldevices. For example, the computing device 1300 may display text,images, etc. by way of the output interface 1312.

It is contemplated that the external devices that communicate with thecomputing device 1300 via the input interface 1310 and the outputinterface 1312 can be included in an environment that providessubstantially any type of user interface with which a user can interact.Examples of user interface types include graphical user interfaces,natural user interfaces, and so forth. For instance, a graphical userinterface may accept input from a user employing input device(s) such asa keyboard, mouse, remote control, or the like and provide output on anoutput device such as a display. Further, a natural user interface mayenable a user to interact with the computing device 1300 in a mannerfree from constraints imposed by input device such as keyboards, mice,remote controls, and the like. Rather, a natural user interface can relyon speech recognition, touch and stylus recognition, gesture recognitionboth on screen and adjacent to the screen, air gestures, head and eyetracking, voice and speech, vision, touch, gestures, machineintelligence, and so forth.

Additionally, while illustrated as a single system, it is to beunderstood that the computing device 1300 may be a distributed system.Thus, for instance, several devices may be in communication by way of anetwork connection and may collectively perform tasks described as beingperformed by the computing device 1300.

Turning to FIG. 14, a high-level illustration of an exemplary computingsystem 1400 that can be used in accordance with the systems andmethodologies disclosed herein is illustrated. For instance, thecomputing system 1400 can be or include the computing system 100.Additionally or alternatively, the computing system 100 can be orinclude the computing system 1400.

The computing system 1400 includes a plurality of server computingdevices, namely, a server computing device 1402, . . . , and a servercomputing device 1404 (collectively referred to as server computingdevices 1402-1404). The server computing device 1402 includes at leastone processor and a memory; the at least one processor executesinstructions that are stored in the memory. The instructions may be, forinstance, instructions for implementing functionality described as beingcarried out by one or more components discussed above or instructionsfor implementing one or more of the methods described above. Similar tothe server computing device 1402, at least a subset of the servercomputing devices 1402-1404 other than the server computing device 1402each respectively include at least one processor and a memory. Moreover,at least a subset of the server computing devices 1402-1404 includerespective data stores.

Processor(s) of one or more of the server computing devices 1402-1404can be or include the processor 102. Further, a memory (or memories) ofone or more of the server computing devices 1402-1404 can be or includethe memory 104. Moreover, a data store (or data stores) of one or moreof the server computing devices 1402-1404 can be or include the datastore 106.

The computing system 1400 further includes various network nodes 1406that transport data between the server computing devices 1402-1404.Moreover, the network nodes 1402 transport data from the servercomputing devices 1402-1404 to external nodes (e.g., external to thecomputing system 1400) by way of a network 1408. The network nodes 1402also transport data to the server computing devices 1402-1404 from theexternal nodes by way of the network 1408. The network 1408, forexample, can be the Internet, a cellular network, or the like. Thenetwork nodes 1406 include switches, routers, load balancers, and soforth.

A fabric controller 1410 of the computing system 1400 manages hardwareresources of the server computing devices 1402-1404 (e.g., processors,memories, data stores, etc. of the server computing devices 1402-1404).The fabric controller 1410 further manages the network nodes 1406.Moreover, the fabric controller 1410 manages creation, provisioning,de-provisioning, and supervising of virtual machines instantiated uponthe server computing devices 1402-1404.

Various examples are now set forth.

Example 1

A computing system, comprising: a processor; a memory that comprises aprimary storage, the primary storage comprises a first subset of recordsof a database, the database being an in-memory database; and a datastore that comprises a secondary storage, the secondary storagecomprises a second subset of the records of the database; the memoryfurther comprises: an update memo that comprises timestamp notices thatspecify statuses of records of the database having updates that span theprimary storage and the secondary storage; and a database managementsystem that is executable by the processor, the database managementsystem processes a transaction, the database management system furthercomprises: a storage interface component that accesses a record of thedatabase for the transaction based on at least one of the timestampnotices of the update memo.

Example 2

The computing system according to Example 1, the memory furthercomprises a private cache reserved for the transaction, the privatecache comprises a copy of the record for the transaction, wherein thestorage interface component reads the record from the secondary storageand stores the copy of the record in the private cache.

Example 3

The computing system according to Example 2, the private cache reservedfor the transaction being separate from a differing private cachereserved for a differing transaction.

Example 4

The computing system according to any of Examples 1-3, the storageinterface component provides a unified interface for accessing therecords of the database, the storage interface component furtherobscures a split of the records of the database between the primarystorage and the secondary storage.

Example 5

The computing system according to any of Examples 1-4, the memoryfurther comprises an access filter that comprises a summary of contentsof at least a portion of the secondary storage, the storage interfacecomponent checks the access filter to identify whether to access thesecondary storage.

Example 6

The computing system according to Example 5, the access filter comprisesat least one of a Bloom filter or a range filter.

Example 7

The computing system according to any of Examples 1-6, the storageinterface component further comprises a control component that managesperformance of an operation upon the record.

Example 8

The computing system according to any of Examples 1-7, the storageinterface component further comprises a migration component thatmigrates the record from the primary storage to the secondary storage,the migration component migrates the record using a first migrationtransaction and a second migration transaction, wherein: duringprocessing of the first migration transaction, the migration component:reads the record from the primary storage; and installs a timestampnotice in the update memo, the timestamp notice comprises an indicatorto ignore a corresponding record in the secondary storage, thecorresponding record pertains to the record; and during processing ofthe second migration transaction, the migration component: inserts thecorresponding record in the secondary storage; updates the timestampnotice in the update memo to remove the indicator to ignore thecorresponding record in the secondary storage; and deletes the record inthe primary storage.

Example 9

The computing system according to any of Examples 1-8, the storageinterface component further comprises a migration component thatmigrates the record while the database is online and active, themigration component at least one of migrates the record from the primarystorage to the secondary storage or migrates the record from thesecondary storage to the primary storage.

Example 10

The computing system according to any of Examples 1-9, the storageinterface component further comprises a deletion component that deletesthe record from the secondary storage, the deletion component: reads therecord from the secondary storage; searches the update memo for amatching timestamp notice for the record; and marks the record logicallydeleted in the update memo responsive to searching for the matchingtimestamp notice, the deletion component one of: updates the matchingtimestamp notice in the update memo for the record, if found; or insertsa new timestamp notice in the update memo for the record, if not found.

Example 11

The computing system according to any of Examples 1-10, the storageinterface component further comprises a read component that reads therecord from the secondary storage, the read component: determineswhether to access the secondary storage for the record using anin-memory access filter; responsive to determining to access thesecondary storage for the record, retrieves the record from thesecondary storage; searches the update memo for a matching timestampnotice for the record; sets timestamps for the record based upon whetherthe matching timestamp notice is found and data indicated by thematching timestamp notice, if found; and inserts the record with thetimestamps into a private cache for the transaction.

Example 12

The computing system according to any of Examples 1-11, the storageinterface component further comprises a cleaner component thatperiodically cleans timestamp notices no longer visible to activetransactions from the update memo.

Example 13

The computing system according to any of Examples 1-12, the storageinterface component further comprises a validation component thatupdates end timestamps of cached records after processing of thetransaction and prior to a beginning of validation of the transaction,the end timestamps being updated based upon timestamp notices thatrespectively correspond to the cached records from the update memo.

Example 14

The computing system according to any of Examples 1-13, the storageinterface component inserts, into the primary storage: a new record; anda new version of the record resulting from an update of the record, thesecondary storage comprises the record.

Example 15

A method of performing one or more operations on records of a database,a memory comprises a primary storage and a data store comprises asecondary storage, the primary storage comprises a first subset of therecords of the database and the secondary storage comprises a secondsubset of the records of the database, the method comprising: duringprocessing of a first migration transaction: reading a first record fromthe primary storage; and installing a timestamp notice in an updatememo, the memory comprises the update memo, the timestamp noticecomprises an indicator to ignore a corresponding record in the secondarystorage, the corresponding record pertains to the first record; andduring processing of a second migration transaction: inserting thecorresponding record in the secondary storage; updating the timestampnotice in the update memo to remove the indicator to ignore thecorresponding record in the secondary storage; and deleting the firstrecord in the primary storage.

Example 16

The method according to Example 15, further comprising deleting a secondrecord from the secondary storage, further comprising: reading thesecond record from the secondary storage; searching the update memo fora matching timestamp notice for the second record; and marking thesecond record logically deleted in the update memo responsive tosearching for the matching timestamp notice by one of: updating thematching timestamp notice in the update memo for the record, if found;or inserting a new timestamp notice in the update memo for the record,if not found.

Example 17

The method according to any of Examples 15-16, further comprisingreading a second record from the secondary storage, further comprising:determining whether to access the secondary storage for the secondrecord using an in-memory access filter; responsive to determining toaccess the secondary storage for the second record, retrieving thesecond record from the secondary storage; searching the update memo fora matching timestamp notice for the second record; setting timestampsfor the second record based upon whether the matching timestamp noticeis found and data indicated by the matching timestamp notice, if found;and inserting the second record with the timestamps into a privatecache.

Example 18

The method according to any of Examples 15-17, the update memo comprisestimestamp notices that specify statuses of records of the databasehaving updates that span the primary storage and the secondary storage.

Example 19

The method of according to any of Examples 15-18, the database being anin-memory database.

Example 20

A method of reading a record of a database as part of processing atransaction, a memory comprises a primary storage and a data storecomprises a secondary storage, the primary storage comprises a firstsubset of the records of the database and the secondary storagecomprises a second subset of the records of the database, the methodcomprising: determining whether to access the secondary storage for therecord using an access filter, the memory comprises the access filter;responsive to determining to access the secondary storage for therecord, retrieving the record from the secondary storage; searching anupdate memo for a matching timestamp notice for the record, the memorycomprises the update memo, the update memo comprises timestamp noticesthat specify statuses of records of the database having updates thatspan the primary storage and the secondary storage; setting timestampsfor the record based upon whether the matching timestamp notice is foundand data indicated by the matching timestamp notice, if found; andinserting the record with the timestamps into a private cache for thetransaction.

As used herein, the terms “component” and “system” are intended toencompass computer-readable data storage that is configured withcomputer-executable instructions that cause certain functionality to beperformed when executed by a processor. The computer-executableinstructions may include a routine, a function, or the like. It is alsoto be understood that a component or system may be localized on a singledevice or distributed across several devices.

Further, as used herein, the term “exemplary” is intended to mean“serving as an illustration or example of something.”

Various functions described herein can be implemented in hardware,software, or any combination thereof. If implemented in software, thefunctions can be stored on or transmitted over as one or moreinstructions or code on a computer-readable medium. Computer-readablemedia includes computer-readable storage media. A computer-readablestorage media can be any available storage media that can be accessed bya computer. By way of example, and not limitation, suchcomputer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM orother optical disk storage, magnetic disk storage or other magneticstorage devices, or any other medium that can be used to store desiredprogram code in the form of instructions or data structures and that canbe accessed by a computer. Disk and disc, as used herein, includecompact disc (CD), laser disc, optical disc, digital versatile disc(DVD), floppy disk, and blu-ray disc (BD), where disks usually reproducedata magnetically and discs usually reproduce data optically withlasers. Further, a propagated signal is not included within the scope ofcomputer-readable storage media. Computer-readable media also includescommunication media including any medium that facilitates transfer of acomputer program from one place to another. A connection, for instance,can be a communication medium. For example, if the software istransmitted from a website, server, or other remote source using acoaxial cable, fiber optic cable, twisted pair, digital subscriber line(DSL), or wireless technologies such as infrared, radio, and microwave,then the coaxial cable, fiber optic cable, twisted pair, DSL, orwireless technologies such as infrared, radio and microwave are includedin the definition of communication medium. Combinations of the aboveshould also be included within the scope of computer-readable media.

Alternatively, or in addition, the functionality described herein can beperformed, at least in part, by one or more hardware logic components.For example, and without limitation, illustrative types of hardwarelogic components that can be used include Field-programmable Gate Arrays(FPGAs), Program-specific Integrated Circuits (ASICs), Program-specificStandard Products (ASSPs), System-on-a-chip systems (SOCs), ComplexProgrammable Logic Devices (CPLDs), etc.

What has been described above includes examples of one or moreembodiments. It is, of course, not possible to describe everyconceivable modification and alteration of the above devices ormethodologies for purposes of describing the aforementioned aspects, butone of ordinary skill in the art can recognize that many furthermodifications and permutations of various aspects are possible.Accordingly, the described aspects are intended to embrace all suchalterations, modifications, and variations that fall within the spiritand scope of the appended claims. Furthermore, to the extent that theterm “includes” is used in either the details description or the claims,such term is intended to be inclusive in a manner similar to the term“comprising” as “comprising” is interpreted when employed as atransitional word in a claim.

What is claimed is:
 1. A computing system, comprising: a processor; amemory that comprises a primary storage, the primary storage comprises afirst subset of records of a database, the database being an in-memorydatabase; and a data store that comprises a secondary storage, thesecondary storage comprises a second subset of the records of thedatabase; the memory further comprises: an update memo that comprisestimestamp notices that specify statuses of records of the databasehaving updates that span the primary storage and the secondary storage;and a database management system that is executable by the processor,the database management system processes a transaction, the databasemanagement system further comprises: a storage interface that migrates afirst record of the database based on at least one of the timestampnotices of the update memo, the first record being migrated from theprimary storage to the secondary storage using a first migrationtransaction and a second migration transaction, wherein: duringprocessing of the first migration transaction, the storage interface: reads the first record from the primary storage; and  installs atimestamp notice in the update memo, the timestamp notice comprises anindicator to ignore a corresponding record in the secondary storage, thecorresponding record pertains to the first record; and during processingof the second migration transaction, the storage interface:  inserts thecorresponding record in the secondary storage;  updates the timestampnotice in the update memo to remove the indicator to ignore thecorresponding record in the secondary storage; and  deletes the firstrecord in the primary storage.
 2. The computing system of claim 1,wherein: the storage interface further accesses a second record of thedatabase for the transaction based on at least one of the timestampnotices of the update memo; and the memory further comprises a privatecache reserved for the transaction, the private cache comprises a copyof the second record for the transaction, wherein the storage interfacereads the second record from the secondary storage and stores the copyof the second record in the private cache.
 3. The computing system ofclaim 2, the private cache reserved for the transaction being separatefrom a differing private cache reserved for a differing transaction. 4.The computing system of claim 1, the storage interface provides aunified interface for accessing the records of the database, the storageinterface further obscures a split of the records of the databasebetween the primary storage and the secondary storage.
 5. The computingsystem of claim 1, the memory further comprises an access filter thatcomprises a summary of contents of at least a portion of the secondarystorage, the storage interface checks the access filter to identifywhether to access the secondary storage.
 6. The computing system ofclaim 5, the access filter comprises at least one of a Bloom filter or arange filter.
 7. The computing system of claim 1, the storage interfacefurther: accesses a second record of the database for the transactionbased on at least one of the timestamp notices of the update memo; andmanages performance of an operation upon the second record.
 8. Thecomputing system of claim 1, the storage interface migrates the firstrecord while the database is online and active.
 9. The computing systemof claim 1, the storage interface further deletes a second record fromthe secondary storage, wherein, to delete the second record, the storageinterface: reads the second record from the secondary storage; searchesthe update memo for a matching timestamp notice for the second record;and marks the second record logically deleted in the update memoresponsive to searching for the matching timestamp notice, wherein, tomark the second record logically deleted, the storage interface one of:updates the matching timestamp notice in the update memo for the secondrecord, if found; or inserts a new timestamp notice in the update memofor the second record, if not found.
 10. The computing system of claim1, the storage interface further reads a second record from thesecondary storage, wherein, to read the second record, the storageinterface: determines whether to access the secondary storage for thesecond record using an in-memory access filter; retrieves the secondrecord from the secondary storage; searches the update memo for amatching timestamp notice for the second record; sets timestamps for thesecond record based upon whether the matching timestamp notice is foundand data indicated by the matching timestamp notice, if found; andinserts the second record with the timestamps into a private cache. 11.The computing system of claim 1, the storage interface furtherperiodically cleans timestamp notices no longer visible to activetransactions from the update memo.
 12. The computing system of claim 1,the storage interface further updates end timestamps of cached recordsafter processing of the transaction and prior to a beginning ofvalidation of the transaction, the end timestamps being updated basedupon timestamp notices that respectively correspond to the cachedrecords from the update memo.
 13. The computing system of claim 1,wherein the secondary storage comprises a second record, and the storageinterface inserts, into the primary storage: a new record; and a newversion of the second record resulting from an update of the secondrecord.
 14. The computing system of claim 1, wherein the storageinterface uses the update memo to maintain transactional consistency ofthe records of the database having updates that span the primary storageand the secondary storage.
 15. A computing system, comprising: aprocessor; a memory that comprises a primary storage, the primarystorage comprises a first subset of records of a database, the databasebeing an in-memory database; and a data store that comprises a secondarystorage, the secondary storage comprises a second subset of the recordsof the database; the memory further comprises: an update memo thatcomprises timestamp notices that specify statuses of records of thedatabase having updates that span the primary storage and the secondarystorage; and a database management system that is executable by theprocessor, the database management system processes a transaction, thedatabase management system further comprises: a storage interface thatdeletes a first record from the secondary storage as part of thetransaction; wherein, to delete the first record from the secondarystorage, the storage interface marks the first record logically deletedusing a timestamp notice in the update memo; and wherein the firstrecord is physically removed from the secondary storage when thetimestamp notice is no longer visible to active transactions from theupdate memo.
 16. The computing system of claim 15, wherein, to deletethe first record from the secondary storage, the storage interfacefurther: reads the first record from the secondary storage; and searchesthe update memo for a matching timestamp notice for the first record;wherein the storage interface marks the first record logically deletedresponsive to searching for the matching timestamp notice; and whereinthe timestamp notice used to mark the first record logically deleted isone of: the matching timestamp notice in the update memo for the firstrecord, if found; or a new timestamp notice in the update memo for thefirst record, if not found.
 17. The computing system of claim 15, thestorage interface further migrates a second record of the database basedon at least one of the timestamp notices of the update memo, the secondrecord being migrated at least one of from the primary storage to thesecondary storage or from the secondary storage to the primary storage,wherein the storage interface uses the update memo to maintaintransactional consistency of the records of the database having theupdates that span the primary storage and the secondary storage.
 18. Thecomputing system of claim 17, the storage interface migrates the secondrecord from the primary storage to the secondary storage using a firstmigration transaction and a second migration transaction, wherein:during processing of the first migration transaction, the storageinterface: reads the second record from the primary storage; andinstalls a timestamp notice in the update memo, the timestamp noticecomprises an indicator to ignore a corresponding record in the secondarystorage, the corresponding record pertains to the second record; andduring processing of the second migration transaction, the storageinterface: inserts the corresponding record in the secondary storage;updates the timestamp notice in the update memo to remove the indicatorto ignore the corresponding record in the secondary storage; and deletesthe second record in the primary storage.
 19. The computing system ofclaim 15, the storage interface further reads a second record from thesecondary storage, wherein, to read the second record, the storageinterface: determines whether to access the secondary storage for thesecond record; responsive to determining to access the secondary storagefor the second record, retrieves the second record from the secondarystorage; searches the update memo for a matching timestamp notice forthe second record; sets timestamps for the second record based uponwhether the matching timestamp notice is found and data indicated by thematching timestamp notice, if found; and inserts the second record withthe timestamps into a private cache.
 20. A computing system, comprising:a processor; a memory that comprises a primary storage, the primarystorage comprises a first subset of records of a database, the databasebeing an in-memory database; and a data store that comprises a secondarystorage, the secondary storage comprises a second subset of the recordsof the database; the memory further comprises: an update memo thatcomprises timestamp notices that specify statuses of records of thedatabase having updates that span the primary storage and the secondarystorage; and a database management system that is executable by theprocessor, the database management system processes a transaction, thedatabase management system further comprises: a storage interface thatreads a record from the secondary storage, wherein, to read the record,the storage interface: determines whether to access the secondarystorage for the record; responsive to determining to access thesecondary storage for the record, retrieves the record from thesecondary storage; searches the update memo for a matching timestampnotice for the record; sets timestamps for the record based upon whetherthe matching timestamp notice is found and data indicated by thematching timestamp notice, if found; and inserts the record with thetimestamps into a private cache.